Facial Expression and Visual Speech based Person Authentication

Most of the person authentication system lacks perfection due to face poses and illumination variation. One more problem in person authentication is the selection of source for feature generation. In this work, videos have been recorded with variations in poses. The videos have been taken in normal office lighting condition. Videos of persons are taken in three situations. First when faces are kept normal, then with smile facial expression and third during speech. Second session of video recording is done similar to the first session with a time gap. This work employs a powerful method to identify the video frames which have single face without pose and excerpt necessary number of frames from the video. Methods are used to automatically identify the mouth area. Features are generated from mouth area in such a way to overcome the issues due to illumination variation. The features created from the first and second session are used to train and test respectively a neural network for person authentication. Among several neural network models, auto associative neural network is used due to its features distribution capturing ability. Person authentication capacity is compared while features created from normal face, features created from smile expression, and visual speech. Equal error rate is used as a tool to compare the capacity of person authentication. The outcome of this project